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WHAT IS CLAIMED IS: 

1 . A computer-implemented method for decomposing images in a sequence of 
images representing a scene, the method comprising: 

transforming images in a sequence of images into a log opponent color 

domain; 

applying a plurality of filters to the transformed images to generate a plurality 
of sequences of filtered images, the plurality of filters including a derivative filter to be 
applied to produce a derivative in a first direction and a derivative filter to be applied to 
produce a derivative in a second direction, where the first and second directions are different 
directions; 

calculating a median image for each of the plurality of sequences of filtered 

images; and 

using the median images to calculate a reflectance image for the sequence of 
images, the reflectance image representing an image the application to which the plurality of 
filters would yield substantially the calculated median images for the sequences of filtered 
images. 

2. The method of claim 1 , further comprising: 

calculating a sequence of illumination images corresponding to the images in 
the sequence of images, such that each image in the sequence of images can be represented 
as the product of the reflectance image and an illumination image of the sequence of 
illumination images. 

3. The method of claim 2, further comprising: 

modifying one or more of the reflectance image or the illumination images; 

and 

combining the reflectance image and the illumination images, including the 
one or more modified images, to create a new sequence of images. 
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4. The method of claim 2, further comprising: 

modifying one or more of the reflectance image or the illumination images; 

and 

using the reflectance and illumination images, including the modified image 
or images, to generate a new image. 

5. The method of claim 1, wherein: 

applying the plurality of filters includes applying each of the plurality of 
filters to each of the luminance and chrominance channels of the opponent color space for 
each pixel in each image in the sequence of images. 

6. The method of claim 1, wherein: 

calculating a reflectance image includes calculating a reflectance image 
having a minimum least squared error between the result of application of each of the 
plurality of filters to the reflectance image and the calculated median image for the 
corresponding sequence of filtered images. 



7. The method of claim 1, wherein: 

applying a plurality of filters includes applying three or more filters to the 
transformed images. 

8. The method of claim 7, wherein: 

the plurality of filters includes one or more derivative filters in addition to the 
derivative filter to be applied to produce a derivative in the first direction and the derivative 
filter to be applied to produce a derivative in the second direction, the additional derivative 
filters being applied to produce a derivative in one or more directions other than the first and 
second directions. 



9. The method of claim 7, wherein: 

the plurality of filters includes filters operable to produce a derivative at a 
plurality of different frequencies. 
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1 0. The method of claim 1 , further comprising: 
aligning the images prior to transforming the images. 

1 1 . The method of claim 1 , further comprising: 

aligning the filtered images prior to calculating the median image. 

12. A computer-implemented method for decomposing images in a sequence of 
images representing a scene, the method comprising: 

transforming images in a sequence of images into a log domain; 

applying a plurality of filters to the transformed images in the sequence of 
images, the plurality of filters including a derivative filter to be applied to produce a 
derivative in a first direction, a derivative filter to be applied to produce a derivative in a 
second direction, where the first and second directions are different directions, and one or 
more additional filters; 

calculating a median image for each of the plurality of sequences of filtered 

images; and 

using the median images to calculate a reflectance image for the sequence of 
images, the reflectance image representing an image the application to which the plurality of 
filters would yield substantially the calculated median images for the sequences of filtered 
images. 

13. The method of claim 12, further comprising: 

calculating a sequence of illumination images corresponding to the images in 
the sequence of images, such that each image in the sequence of images can be represented 
as the product of the reflectance image and an illumination image of the sequence of 
illumination images. 

14. The method of claim 1 3 , further comprising: 

modifying one or more of the reflectance image or the illumination images; 

and 

combining the reflectance image and the illumination images, including the 
one or more modified images, to create a new sequence of images. 
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1 5 . The method of claim 1 3 , further comprising: 

modifying one or more of the reflectance image or the illumination images; 

and 

using the reflectance and illumination images, including the modified image 
or images, to generate a new image. 



16. The method of claim 12, wherein: 

calculating a reflectance image includes calculating a reflectance image 
having a minimum least squared error between the result of application of each of the 
plurality of filters to the reflectance image and the calculated median image for the 
corresponding sequence of filtered images. 

17. The method of claim 12, wherein: 

the one or more additional filters include one or more derivative filters in 
addition to the derivative filter to be applied to produce a derivative in the first direction and 
the derivative filter to be applied in the second direction, the additional derivative filters 
being applied to produce a derivative in one or more directions other than the first and second 
directions. 

18. The method of claim 12, wherein: 

the one or more additional filters include filters operable to produce a 
derivative at a plurality of different frequencies. 

19. The method of claim 12, further comprising: 

aligning the images prior to transforming the images into the log domain. 

20. The method of claim 12, further comprising: 

aligning the filtered images prior to calculating the median image. 
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21 . A computer program product embodied on an information carrier for 
decomposing images in a sequence of images representing a scene, the computer program 
product comprising instructions operable to cause a computer system to: 

transform images in a sequence of images into a log opponent color domain; 

apply a plurality of filters to the transformed images in the sequence of 
images, the plurality of filters including a derivative filter to be applied to produce a 
derivative in a first direction and a derivative filter to be applied to produce a derivative in a 
second direction, where the first and second directions are different directions; 

calculate a plurality of median images, one for each sequence of filtered 

images; and 

calculate a reflectance image for the sequence of images, the reflectance 
image representing an image the application to which the plurality of filters would yield 
substantially the calculated median image for the sequence of filtered images. 

22. The computer program product of claim 2 1 , further comprising instructions 
operable to cause a computer system to: 

calculate a sequence of illumination images corresponding to the images in 
the sequence of images, such that each image in the sequence of images can be represented 
as the product of the reflectance image and an illumination image of the sequence of 
illumination images. 

23 . The computer program product of claim 22, further comprising instructions 
operable to cause a computer system to: 

modify one or more of the reflectance image or the illumination images; and 
combine the reflectance image and the illumination images, including the one 
or more modified images, to create a new sequence of images. 

24. The computer program product of claim 22, further comprising instructions 
operable to cause a computer system to: 

modify one or more of the reflectance image or the illumination images; and 
use the reflectance and illumination images, including the modified image or 
images, to generate a new image. 
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25 . The computer program product of claim 2 1 , wherein: 

the instructions operable to cause a computer system to apply the plurality of 
filters include instructions operable to cause a computer system to apply each of the plurality 
of filters to each of the luminance and chrominance channels of the opponent color space for 
each pixel in each image in the sequence of images. 

26. The computer program product of claim 21, wherein: 

the instructions operable to cause a computer system to calculate a reflectance 
image include instructions operable to cause a computer system to calculate a reflectance 
image having a minimum least squared error between the result of application of the plurality 
of filters to the reflectance image and the calculated median image for the sequence of 
images. 

27. The computer program product of claim 21, wherein: 

the instructions operable to cause a computer system to apply a plurality of 
filters include instructions operable to cause a computer system to apply three or more filters 
to the images in the sequence of images. 

28. The computer program product of claim 27, wherein: 

the three or more filters include one or more derivative filters in addition to 
the derivative filters applied to produce a derivative in the first and second directions, the 
additional derivative filters being applied to produce a derivative in one or more directions 
other than the first and second directions. 

29. The computer program product of claim 27, wherein: 

the three or more filters include one or more filters operable to produce a 
derivative at a plurality of different frequencies; 

30. The computer program product of claim 2 1 , further comprising instructions 
operable to cause a computer system to: 

align the images prior to transforming the images into the log opponent color domain. 
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3 1 . The computer program product of claim 2 1 , further comprising instructions 
operable to cause a computer system to: 

align the filtered images prior to calculating the median image. 

32. A computer program product embodied on an information carrier for 
decomposing images in a sequence of images representing a scene, the computer program 
product comprising instructions operable to cause a computer system to: 

transform images in a sequence of images into a log domain; 

apply a plurality of filters to the transformed images in the sequence of 
images, the plurality of filters including a derivative filter to be applied to produce a 
derivative in a first direction, a derivative filter to be applied to produce a derivative in a 
second direction, where the first and second directions are different directions, and one or 
more additional filters; 

calculate a plurality of median images, one for each sequence of filtered 

images; and 

calculate a reflectance image for the sequence of images, the reflectance 
image representing an image the application to which the plurality of filters would yield 
substantially the calculated median image for the sequence of filtered images. 

33. The product in claim 32, further comprising instructions operable to cause a 
computer system to: 

calculate a sequence of illumination images corresponding to the images in 
the sequence of images, such that each image in the sequence of images can be represented 
as the product of the reflectance image and an illumination image of the sequence of 
illumination images. 

34. The computer program product of claim 33, further comprising instructions 
operable to cause a computer system to: 

modify one or more of the reflectance image or the illumination images; and 
combine the reflectance image and the illumination images, including the one 
or more modified images, to create a new sequence of images. 
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35. The computer program product of claim 33, further comprising instructions 
operable to cause a computer system to: 

modify one or more of the reflectance image or the illumination images; and 
use the reflectance and illumination images, including the modified image or 
images, to generate a new image. 

36. The product in claim 32, wherein: 

the reflectance image has a minimum least squared error between the result of 
application of each of the plurality of filters to the reflectance image and the calculated 
median image for the corresponding sequence of filtered images. 

37. The computer program product of claim 32, wherein: 

the one or more additional filters include one or more derivative filters in 
addition to the derivative filters applied to produce derivatives in the first and second 
directions, the additional derivative filters being applied to produce derivatives in one or 
more directions other than the first and second directions. 

38. The computer program product of claim 32, wherein: 

the plurality of filters includes one or more filters operable to produce a 
derivative at a plurality of different frequencies. 

39. The computer program product of claim 32, further comprising instructions 
operable to cause a computer system to: 

align the images prior to transforming the images into the log domain. 

40. The computer program product of claim 32, further comprising instructions 
operable to cause a computer system to: 

align the images prior to calculating the median image. 
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41 . A method for extracting the constant reflectance component from a sequence 
of images of a natural scene comprising: 

capturing a sequence of images of a scene; 

aligning each of the images of the sequence of images; 

transforming each of the images in the sequence of images into a log opponent 

color domain; 

applying a plurality of filters to the images in the sequence of images, the 
plurality of filters including a derivative filter to be applied to produce a derivative in a first 
direction, a derivative filter to be applied to produce a derivative in a second direction, where 
the first and second directions are different directions, and one or more additional filters; 

calculating a median image for each sequence of filtered images; and 
calculating a reflectance image for the original sequence of images, the 
reflectance image having a minimum least squared error between the result of application of 
each of the plurality of filters to the reflectance image and the calculated median image for 
the corresponding sequence of filtered images. 
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